Papers with Contextualized word representations
On the Systematicity of Probing Contextualized Word Representations: The Case of Hypernymy in BERT (2020.starsem-1)
Copied to clipboard
| Challenge: | Existing studies have found that BERT can correctly retrieve noun hypernyms in cloze tasks, but this does not correspond to systematic knowledge in BERT. |
| Approach: | They propose to use BERT to probe for hypernymy knowledge encoded in representations for cloze tasks to find out whether it is systematic or not . |
| Outcome: | The proposed model can retrieve hypernyms in cloze tasks, but not systematic knowledge in BERT. |
Verb Sense Clustering using Contextualized Word Representations for Semantic Frame Induction (2021.findings-acl)
Copied to clipboard
| Challenge: | Contextualized word representations are effective in many natural language processing tasks, but it remains unclear to what extent they can cover hand-coded semantic information such as semantic frames. |
| Approach: | They compare contextualized word representations with two English frame-semantic resources . they find that several contextualized representations are informative for semantic frame induction . |
| Outcome: | The proposed representations are useful in natural language processing tasks, but are not fully understood by the literature. |
Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations (D19-1)
Copied to clipboard
| Challenge: | Contextualized word representations are effective in downstream tasks such as question answering, named entity recognition, and sentiment analysis. |
| Approach: | They propose to integrate pre-trained contextualized word representations into a neural network that captures the whole sentence and the word representation in the sentence. |
| Outcome: | The proposed approach outperforms the state-of-the-art approach that makes use of non-contextualized word embeddings on multiple benchmark WSD datasets. |